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ABSTRACT 

A Study designed to serve as an initial step in 
building a theory of coapute^r literacy was conducted to provide nev 
inforaation concerning hov huaans think about calculators and to 
deter Bine hov individual di«:ferences in students* intuitions affect 
their underattanding and use of the calculator. Thirty^three expert 
and 33 novice calculator users vere asked to fill out guestionnalres, 
and then to predict vhat nuaber vould be in the display of the 
calculator after a series of key presses for 88 aath probleas. 
Analytic tech^iigues of cognitive psychology vere applied to aasess 
the perforaance df the subjects coapared to the perforaance of the 
calculators, as veil as subject perforaance in light ox the type of 
calculator ased# and the aaount of previous experience vlth 
calculators. Results indicated that there vere treaendoas individual 
differences aaong users in their interpretations of the logic of the 
calculator's operating systea. Future vork is recoaaendec^ to 
deter sine vhether intuitions, once diagnosed, can be alteied through 
instruction, and to deteraine vhether ^ople vith certain intuitions 
can use their calculators aore creatively c learn a new coaputer 
language aore efficiently than people vith other sets of intuition* 
Rlneteen references are listed and supporting data are appended. 
(HER) 
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Abstract 

Thlrty<-three novice and 33 expert users predicted what number would be In the 
display of a calculator after a sequence of key presses (such as 2 + 3 +). The 
performance of each subject on 88 problems was formally described as a 13 line 
production system. Condition^ were key presses (such as + after a ntmber); 
actions were changes In the display or Internal registers (such as display the 
evaluation of the expression In the register). Large Individual differences 
were observed. Differences among subjects Included when an expiesslon Is eval- 
uated and displayed (e.g., after a + key, x key, « key, and/or number key), 
whether or not the display Is Incremented when two operation keys were pressed 
In 8equenc^(e.g., 2 + + or 2 x x), whether or not the display is incremented 
when an equals was pressed after an operation (e.g., 2 + •), what the order of 
arithmetic would be in a chain (e.g., 2 4 3x7). Experts were more consistent 
in their performance and tended to be more likely than novices to base answers 
on standard operating systems. Implications for developing a theory of com- 
puter literacy were discussed. 
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Analysis of Students' Intuitions About the Operation of Electronic Calcuators 

-« If 

I », 
> \ ^ 

Within the past decade electronic calculators havlvi'^come a part of our 
society. Including widespread and rapid acceptance in schools (Mulllsh, 1976). 
Based on a survey of articles .and editorials published \^thln the past few 
years In Arithmetic Teacher an d Mathematics Teacher , as well as a policy state- 
ment by the National Council of Teachers of Mathematics (1976), It Is clear 
that calculators will play an Important role In the education of our children. 
For example, the following statements by mathematics educators are typical; 
"All students will profit from having access to a calculator" C^awronskl & 
Coblentz, 1976). "I propose that we make fulle3t possible us*» of calculators 
In all grades of our school" (Hopkins, 1976). "Not since the printing press 
has any Invention had such potential for revoluntlonlzlng education, particu- 
larly mathematics education" (Rudnlck & Krullk, 1976). a- 

However, In spite of these optimistic predictions and endorsements, the 
research community has been slow In providing Information that might be useful 
In this Impending calculator-curriculum revolution. For example, most experi- 
mental studies to date have compared changes In achievement and/or attitude 
scores for students who use calculators In school versus students who were not 
allowed to use calculators (Gastln, 1975; Roberts & Fabrey, 1978; Roberts & 
Glynn, 1979; Schnur & Land, 1976; Suydam, Note 1). In a recent review of 34 
studies, mos*- of which were not published In journals, Roberts (1980) observed 
that there was clear evidence that calculators improve computational efficiency 
but no concensus concerning effects on higher level conceptuai-achievement-or— 
attitudes towards mathematics. Thus, he concludes that "the research literature 
offers no guidance" concerning how to incorporate calculators into school curricula. 
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The present paper does not attempt to address the question of whether cal- 
culators Influence changes In mathematics achievement and attitude scores* 

o 

Rather, this paper. Is based on the Idea that since calculators will become a 
part of everyday life for students. It Is Important to know ho^ people come to 
understand and Interact with calculators. The calculator represents a student ^s 
first Introduction to a computer,- to a computer language, and to computer lit- 
eracy In general. Pressing a key Is analogous to a computer command. In spite 
of tremendous breakthroughs In development of Improved calculator hardware for 
the mass market, there ha& been comparatively little work on what Shnelderman 
(1980) calls "software psychology". That Is to say, we know very little about 
how people 'understand calculators, what types of Instruction will help people 
become creative users of calct;lators, why some people seem to not use them very 
well, or how to design operating systems that make psychological sense. 

Since calculator useage seems so simple and since even children can teach 
themselves to use a calculator In a short time, some educators have suggested 
that explicit Instruction or concern about users* understanding of calculators 
is not needed (Bell, 1976). This might be correct If one^s goal Is simply to 
have students use the calculator as a "black box" that gives answers for mundane 
computation. However, when the goal Is to promote productive problem solvers, 
there Is reas^^n to believe that tha student *s understanding of how the calcu- 
lator operates Is Important. For example, Scandura, Lowere, Veneskl & Scandura 
(1976) found that some students who are left to teach themselves develop blzzare 

Intuitions; for example, some subjects concluded that the plus (+) and equals 

« 

(•) key did nothing since they caused no visible change in the display. On the 
other hand, Mayer (1980) found that by letting fourth-graders explore the 
functions of the operation ^csys some of them discovered that pushing the same 
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operation key more than once would cause the calculators to repeat the process 
with the number last entered. Leaving the understanding of the basic functions 
of the calculator to chance might create as many students with incorrect in- 
tuitions as correct ones. Thus, although two users may be able to use the 
calculator to solve basic computational problems, there may be large individual 
differences in the way users understand and interact with calculators. 

Elucidation of individual differences in students' intuitions about the 
operation of calculators (i.e.,, students^ conception of what goes jn inside the 
"black box" whefi a key is pressed) is the logical first step in building a / 
theory of computer literacy, and is the goal 'of the present study. In par- 
ticular, this study addresses two related issues. 



(1) What knowledge do people have about how calculators work? Since most 
users are "self-taught" and seem to be able to use their calculatots, an im- 
portant issue concerns what they have learned. Sinc^some intuitions may lead 
to more creative use of calculators and to better transfer to computer languages 
(such as progranmble calculators or BASIC) .it would be useful to be able to 
describe exactly whitt people's intuitions are. Recent research on the cognitive 
analysis of computational skill suggests that two children with the same ap- 
parent performance may be using entirely different conceptions of computation. 
For example, Groen & Parkman (1972) have developed cognitive models addition, 
and Woods, Reanick & Groen (1975) have developed models of subtraction. More 
recently. Brown & Burton (1975) have developed a BUGGY , program which serves to 
diagnose problems in a child's arithmetic procedures by developing a formal 
deacript-Jon of the algorithm that the child is using. Successful application of 
cognitive analysis tools to the problem of describing computational skill en- 
courages the idea that similar techniques can be used to formalize how students 
understand calculator logic. 
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(2) What are the differences in intuitions among individuals? For ex- 
ample, do "experts" have different intuit-.on8 than "novices?" In recent studies, 
Larkln (1979) and Simon & Simon (1978) have been able to formalize differences 
between what experts and novices know about solving physics problems. Larkin 
(1979) has been able' to represent the differences in terms of differences in the 
organization and size of productions in a production system. Similar tech- 
^niques may be applied to representing differences between experts and novices in 
the domain of calculator ude. 

1^ STUDIES 1 AND 2 

The purpose of study 1 was to determine the knowledge that ordinary users 
have concerning the operation of hand-held calculators. In particular, the goal 
was to formally describe each subject's conception of the calculator's operating 
system. The purpose of study 2 was to determine whether the formal descriptions 
developed in study 1 would also describe the conceptions of people who were more 
knowledgeable about operating systems. For purposes of this paper, subjects in 
study 1 are labeled "novices" and subjects in study 2 are labeled "experts". 

Method 

Subjects 

( 

The subjects in study 1 were 46 college undergraduates recruited from the 
subject pools at the University of Pittsburgh and the University of California, 
Santa Barbara. All subjects participated in order to fulfill a requirement for 
their introductory psychology courses. Subjects in study 1 had no previous 
experience with computer programming nor with the concept of operating systems. 
Thirteen of the 46 subjects in study 1 produced inconsistent performance, so ' 
only dafa for the remaining 33 subjects was used for the analyses. Of these 33 
subjects there. were 16 females and 17 males, and 26 of the subjects owned a 
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calculator* The mean age was 18 years, the mean SPA was 3*0, the mean SAT- 
Qxxantitatlve score was 547, the mean SAT-Verbal score was 491. 

The subjects In studyf 2 were 35 college undergraduates recruited from a 

* 2 
course in computer programming at the University of California, Santa Barbara. 

All subjects had taken at least one previous course in computer programming and 
were currently in a course that included study of operating systems. Of these 
35 subjects, one gave Inconsistent performance and one failed to follow direc- 
tions. The remaining 33 subjects were retained for analysis in this study. 
There were 13 feoAles and 20 males, and 32 of the subijects owned a calciUator* 
The mean age ^a^s 21 years, the mean CPA was 2.9, the mean SAT-Quantitative score 

r 

was 669, and the mean SAT-Verbal score was 552. 

The main difference between subjects in study 1 (novices) and study 2 
(experts) is that all of the experts had formal instruction in computer pro- 
gramming and some introduction to operating systems while none of che novices 
did; the experts were older, t(60) - 2.66, £ < .01;/and the experts had higher 
SAT^Quantitative scores, t(44) » 4.67, £ < .001. Thua, while the main focus was 
on comparing "liberal arts" students who had no formal programming experience to 
"engineering" students who had formal training in programming, any comparisons 
between th«^ subjects in the two studies mu^t be made in light of other dif- 
ferences such as age and SAT scores. 
Matferials 

The materials in study 1 and study 2 were essentially identical. Materials 
consisted of a questionnaire, an instruction sheet, and two two-page problem . 
sets. 

The questionnaire was an 84 x 11 inch sheet of paper which asked the 
subject to^ indicate his or her age, sex, CPA, SAT scores, year in school, major 
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In school, experience with computer programming, experience with calculators, 
and previous mathematics courses. In particular, subjects were asked to In* 
dlcate whether they owned or regularly used a particular calculator, an^ If so, 
to give the name of the model. In addition, subjects were asked to check a box 
corresponding to average number of minutes per week they used a calculator-- 
less than 10, 10 to 30, 30 to 60,' more than 60. 

The Instructions for the problems were typed onto an 8i x 11 Inch sheet of 
paper. Instructions described a typical calculator and the task. 

Each of the two problem sets consisted of 44 problems typed onto two 6i 
X 11 Inch sheets of paper with one problem per double-spaced line. Each problem 
presented a series of key strokes and provided a blank space for the subject to 
indicate what number would be In the display* Each problem contained from one 
to seven key strokes and each key stroke was either a single digit (2, 3 or 7), 
a plus key (+) , a multiply key (x) , or an equals key (•) . X^e two problem sits 
(Set A and Set B) provided for a reliability check since each problem In Set A 
corresponded to a problem of the same form In Set B, and both sets presented the 
corresponding problems In the same order. However^ the specific digits used In 
corresponding problem were different. For example, problems 2+3-P7 or 2+3x or* 
2 »-i "»- In set A corresponded to 7+3^1-2 or 7+3x or 7+-+— H-, respectively. In Set 
B. The complete list of Set A problems Is ^Iven In the left side of Table 1. 
Procedure 

The procedures were essentially Identical In study 1 and study 2 except 
than subjects were run Individually in study 1 and were run as a group in 
study 2. 

First, each subject filled out the questionnaire. Then the instructions 
ware presented. Subjects were asked to suppose that they had Just been given a 
o«w seandard four-function calculator that worked efficiently, and to suppose 
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that they would be using this calcul'atox^ throughout the session. They were told 
that fcr each ptoblem, their job was to predict what number would be In the 
display of the calculator after the series of key presses (assuming the calcu- 
lator was cleared at the beginning of the problem) • Then, one of the problem 
sets was randomly selected and given to the subject; when the subject finished * 
the first set« the other set was given. Subjects were asked to put their 
answers In the space next to each problem and there was^n^t'lme limit. 

Results and Discussion 

Scoring 

The data for each subject In each study consisted of a number (I.e., the 
subject's answer) for each of the 88 problems. 
Reliability of Performance 

Since two forms of the same 44-problem test were administered to each 
subject, It Vas possible to determine the reliability of each subject's per- 
formance. For each of the possible answers to the 44 problems In set A, 
corresponding answers were generated for set B. For example. If a subject gave 
12 as the answer for 2+3+7 In set A, the corresponding answer for 7+3+2 In 
set B would also be 12; If a subject gave 7 as the answer for the above prob- 
lem In set A, the corresponding answer In set B would be 2. Similarly, If a 
subject gave 2 as the answer for,. 2+«+*+« In set A, the corresponding answer for 
7l"**> * In set B Is 7; If a subject gave 16 for the above problem In set A, the 
corresponding answer for set B Is 56. Reliability scoring was conducted by 
matching each of the 44 problems In set A with Its corresponding problem In set 
B; If the answers did not correspond, subjects were given a point. 

Thirty-three of the 46 subjects In study 1 displayed six or less (I.e., 
less than 14%) non-matching scores between set A and set B. Data for the 13 



ERJC 



Intuitions About Calculators 
9 

subjects who displayed more' than six non-matched (i«e., over 14% unreliable 
answers) wt re not analyzed further. The mean non-matching score for ail 46 
subjects in study 1 was 5! 3 or 12%; the mean non-matching score for the 33 
selected subjects in study 1 was 2,6 or^ 6%. 
^ In study 2, one of the 35 subjects gave more than six- non-matchinc answers 

/ 

between set A and set B. Data for this subject as well as ^ one subject who . 
failed to follow directions were not included in subsequent analyses. The mean 
numb^ of non-matching answers for the 33 selected subjects in study 2 was .8 or 
- 1%. . . ,j ' ' 

One question that may be raised at this point is whether the experts and 
novices differ with respect to the reliability of their performances,. The, 
' proportion of unreliable novices (13. out of 46) was significantly higher than 
the proportion of unreliable experts (1 out of 35) as determined by a chi- 
square test, • 7,28, df ^.1, £ < ,.01, In addition, for the 33 selected 
subjects in each study, the novices produced significantly more unreliable 
answers than the experts as determined by a t-test, £(64) « 10,59, £ < ,001. ' 
Thus, as might be expected, experts were more consistent in the way they an- 
^ swered problems than were the novices. 

All subsequent analyses are based on answers to set* A- for the 33\(ubjects 
in each group 

Performance of Subjects Compared to Perfongance of Calcixlators 

In this section we address the question of which calculator mo^t closely 
fits the answers given by the subjects, * pt the 33 subjects in study 1, 17 owned 
simple Texas Instruments (TI) models, th]^ owned Sharp models, one Iwned a 
Rockwell model, one o^ed a Hewlett-Packard HP-21, and eleven either did not 
own a calculator or could not remember what kind "they owned. 
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Since TI, Sharp and Rockwell were the relevant models owned by subjects In 
study 1, answers to each of the 44 problems in set A were generated using each 
of the three brands of calculator. Interestingly, while most of the calculators 
gave identicscl answers for most problems, there were different answers produced^ 
by at least two of the calculators^ on 20 of the 44 problems. 

A difference score was computed for each subject for each of the three 
calculator brands. The, difference score was based on tallying, the number of 
times that the subject's answer was not identical to the calculator's answer for 

the 4A problems in set A. Mean difference scores in study 1 were 8.8 (?0%) for, 

\ 

Tl, 20.8 (47%) for Rockwell, ait^ 14.9 (34%) for Sharp. A one-way analysis 
of variance was conducted on the difference scores with brand of calculator as a 
within subjects factor. The ANOVA revealed that the difference scores listed 
above were significantly different from one another, F (2,64) • 42.6^ £ < .001. 
Supplementary Newman-KeuJs tests indicated that the score for TI was signifi- 
cantly better than for Rockwell, but no other differences were significant 
(£ < .05) . 

Of the 33 subjects in study 2, 23 owned Texas Instruments (TI) models, two 
owned Sharp models, three owned Casio models, three owned Hewlett-Packard, and 
two either did not own a calculator or could not remember what kind they owned. 
Thus, as with novices the most freqpuntly owned calculator was TI. However, 
II4 of the 33 novices did not own or could not remember which calculator they 
owned, whlle^only 2 of the 33 experts fell into this category. According to a 
chl-square test, this difference between proportions is significant, 
• 6.13, df - 1, £ < .025. 

The mean difference s^.ores in study 2 were 90 (20%) for TI, 18.2 (41%) 
Rockwell, and 13.7 (31%) for Sharp. A one-way analysis of variance was 

/ 

I 
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conducted on the dirfference scores with brand of calculator as a within subjects 

L ^ f 

factor. The ANOVA Tevealed chat the difference scores listed above were slg- ( 
niflcantiy different from one another, F (2,64) - 23.74, £ < .001. Supple- 
mentary Newman-Kupls tests revealed that the score for TI was significantly 
better than for Rockwell or for Sharp, and that the ora for Sharp was signifi- 
cant?-/ better than for Rockwell <£ < .t)5). ' . 

Thus, both In study 1 and In study 2, there Is evidence that TI's o^ratlng 

f\ j) 

system most closely matches the Intuitions of subJe^Qts for the 44 f^obl^^ In 
the test. In comparing experts and novices, there is" no evidence of^bi^^if* " 
ferences. In which calculator gives the best fit; t-tests revealed no differences 
between experts and novices with respect to their scores for TI, 54) < 1, for 
Rockwell, t(64) • 1.58, or for Sharp, t^(64) » 1.27. 
Performance of Subjects By Type of Calculator They Own 

The previous section suggested that subjects In our sample generated per- 
formance that more closely matched the performance of a TI calculator than the 

o — - . _ . - . 

other calculators we tested. However, since the TI Is the brand of calculator 
that most subjects In our sample owned, tha above results may be mainly due to 
experience with TI calculators. In order to test this Idea, subject? In study 1 
were divided Into two groups: those who owned a TI calculator (n « 17) and 
those who do not (n • 16). 

The mean, difference scores for the Tl-owners lii study 1 were : TI «8.0, 
Rockwell - 20.0, and Sharp - 14.1; the mean difference scores for the non-TI 
ownert were: TI - 9.8, Rockwell « 21.6, Sharp - 15.8. As can be seen, for both 
Tl-owners and non Tl-owners, the difference scores are least for TI. An ana- 
lysis of variance was conducted on the difference score data with group as a 
between subjacts factor and type of calculator as a within subjects factor. 
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There was a significant ^fference between the calcula^^ors.in how well they 
matched the performance of all subjects, F (2,60) - 51.16, £ < .001, but there 
.as no group x calciilator interaction i F (^,60 < 1. Thus, there was no evidence 
,that the non-TI owners were different from TI owners with respect to their 
performance being best fit by a TT calculator. A separate one-way ANOVA was 
conducted on the data for the non-owners with type of calculator as a wlthin- 
subjects factor. The differences in difference scores were significant, F (2,30) 
■ 37.57, p < .001; subsequent Newman-Kuels tests showed that the score for the 
TI were significantly better than for Sharp or Rockwell^ among non-TI owners 
(£ < .05). 

^ As an additional analysis, the performance of each subject was labeled as 
Tl-like, Rockwell-like or Sharp-like, b^sed on which of the three calculators 
produced the lowest difference score for tl^ft^subject. Of the 33 subjects in ^ 
study 1, 29 were classified as Tl-like and 4 were better matched with other 
brands. For the Tl-owners, 15 were classified as Tl-like and 2 were best fit by 
other brands; for the non-TI owners, 14 were classified as Tl-like and 2 were 
best fit by^ other brands. A Fisher's Exact test revealed that there was no 
•evidence of any differences amonp. the two groups (TI owners versus non-owners) 



in thi proportion of Tl-like . v* s. 

The performance of the experts in study 2, like the novices in study 1, 

most closely matches the performance of the TI, but this may be due to the fact 

that TI is the jjost prevelant calculator used among the experts. As in study 1, 

4 

this idea was tested by dividing the experts into- those who owned a TI calcu- 
lator (n " 22Kand those who did not (n - 11). The mean difference scores for 
the Tl-owners were: TI • 76, Rockwell - 34.4, Sharp - 13.2; the main difference 
, scores for the non-TI owners were: TI - 11.8, Rockwell - 17.4, Sharp - 14.6. An 
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analysis of variance was conducted on the difference score data witn ownership 
group as a between subjects factor and type of calculator as a within subjects 
factor. As expected, there was a signi^cant difference among ttre calculators 
in how well they njatched the performance of all subjects, F (2,60) • 19.21, 
£ < .01; however, as in study 1, jhere was no interaction between group and 
calculator, F (4,60) < !• Thus, as in study 1, there was no evidence that non- 
TI owners were different from VI owners with respect to their performance being 
best fit by a TI calculator. 

As an additional analysis, the performance of each subject in study 2 was 
labeled as Tl-like, Rockwell like, or Sharp-like, based on which of the three 
calculators produced the lowest difference scores for each subject. Of the 33 
subjects, 26 were classified as Tl-like and 7 were better matched by other 
brands. For Tl-owners, 18 were classified as Tl-like and 4 were best fit by 
other brands; for the non-TI owners, 8 were cla99ified as Tl-like and 3 were 
best fit by other brands. A Fisher's Exact test revealed that there was no 
evidence of any differences among the two groups (TI owners versus non-owners) 
in the proportion of Tl-like subjects. 

This sect;ion helps clarify the earlier finding that subjects* performance 
is most closely matched by TI's operating system. Since this finding seems to 
be present for both Tl-owners and non-owners it may be attributed to the '^in- 
tuitive appeal" of the II operating system rather than to users having more 
experience with TI products. However, it should be pointed out that the corres- 
pondence between the calculator's answers and the subjects' answers are far from 
perfect, even when we choose the best fitting calculator. 
Performance of Subjects by Amount of Experience With Calculators 

This section explores the issue, of whether*subjects who differ with respect 
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to hov much they usa a calculator also differ with respect to Intuitions about 
the operating system underlying *tb*i calculator. In order to address this Issue, 
subjects In study 1 were divided into two groups based On their reported weekly 
use of calculators: low— less than 10 minutes per week (n • 17), and moderate — 
10 or more minutes per week (n - 16} . 

The aean difference scores for study 1 on each of the three calculators 
' brands was TI « 10.2, Rockwell * 22.2, and Sharp - 16.5 for the low experience 
group, and TI - 7.4, Rockwell • 19.3, and Sharp « 13.3 for the moderate^ ex- 
perience group. As can be seen, TI produces the lowest score (i.e., best fit) 
for subjects* performance in both groups. An analysis of variance was conducted 
on the difference score data with experience (low vs. moderate) as a between 
subjects factor and calculator brand as a within subjects factor. As in prevlou 
analyses, there was a significant overall difference among the caldulators in 
how closely they fit the intuitions of the subjects, F (2,62) - 40.62, £ < .001, 
and there was no interaction between group and calculator, £ (2,62) < 1. Thus, 
there was no evidence in study 1 that the superior fit of the TI calculator was 
influenced by how much experience a subject had. 

As in the previous section, each subject was classified as being either TI- 
like, Rockwell-like, or Sharp-like based on which calcula^toi^roduced the least 
number of differences with the subject's actual performance. ^*or^he low 
experience group in study 1, 15 subjects were best fit by TI and 2 were best fit 
by another calculator; for the moderate experience group 14 were best ^it by TI 
and 2 were best fit by another calculator. A Fisher's Exact test showed that 
there were no significant differences between low experience and moderate 
experience groups with respect to the proportion of Tl-like subjects. Thus, 
there is no evidence in this analysis that amount of experience with calculators 
influence the subjects' IncuitionQ about the operating systems of calculators. 

/ 
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In study 2, eight subjects were In the low experience category and 25 In- 
dicated moderate experience. Corresponding analyses were conducted on the data 
for the experts In study 2. The mean difference scores for study 2 were: TI > 
^^V5> Rockwell ■ 18.1, and Sharp - 14.4 for the low experience group, and TI • 8.8, 
Rockwell - 18.3, and Sharp - 13.5 for the moderate experience group. An ANOVA 
revealed the expected overall effect, F (2,52) » 18.92, £ < .001; but thera was 
no interaction between experience group and calculator, F (2,62) < 1, and thus 
no evidence that the superior fit of the TI calculator was influence^ by^ how 
much experience a subject had with calculators. For the low experience group 
in study 2, 6 of the 8 subjects vere best fit by TI while for .the moderate 
experience group, 21 of the 25 subjects were best fit by TI. A Fisher's Exact 
test shows there is no significant difference with respect to the proportion of 
Tl-like subjects among low and moderate experience subjects in study 2. Thus, 
for both experts and novices, there is no evidence that experience with calcu- 
lators influence? the subjects' intuitions about the operating system of calcu- 
lators. 

Inter-Subject Consistency in Performance 

The foregoing analyses indicates that subjects' performance was closest to 
that of a TI calculator, and ^t this pattern was not influenced by whether 
subjects actually owned a TI calculator nor whether they had experience with 
using a calculator. However, the foregoing analyses also made clear that sub- 
jects' performance could not be adequately described as corresponding to one 
particular calculator's performance, since the best fitting calculator predicted 
only about 80Z of the answers. .In this section, we explore the question of how 
slallar or different the subjects' answers were from subject to subject, ^ 
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Table 1 gives a list of the 44 problems In set A as well as the answers 
given by subjects In study 1 and study 2. Each answer that was given by any 
subject Is listed (In parentheses) along with the number of subjects who gave 
that answer In study 1 and In study 2. As can be seen, for each question there 
la an answer that occurs most often (I.e., the modal answer) and there may be 

one or more other answers given by some subjects (I.e., alternative answers). 

'I 

The percentage of subjects' answers that are modal answers. I.e., answers that 
correspoi^d to the most common answer for each question Is 83Z for study 1 and 
81Z for study 2. A t-test revealed that the experts and novices do not differ 
with respect to the percentage of modal answers, t(64) < 1* \ Kolmogorov-Smlmov*s 
One-Samt{jLe tests based on the data for both'studles together^ Indicated that the 
follo%d.ng problems produce significant (£ < ,05) number of non--model answers: 
4, M, 15, 17, 19, 21, 23, 24, 25, 27, 28, 29 and 31 through 44. Thus, the 

dat#in Table 1 encourages the conclusion that there are substantial Individual 

f 

differences among subjects In their Intuitions about calculators. In order to 
more closely examine and describe these differences, all subsequent analyses 
will involve descriptions of single subjects rather than group data. 



Insert Table 1 about here 



Analysis of Performance of Individual Subjects on Simple -Problems 

The goal of the analysis Iq this section is to provide a formal description 
of the knowledge that each subject has concerning how the calculator solves 
slfliple problems. Thus, for each subject, a model was developed which could 
generate the subject's answers on simple test problems. This section describes 
the data source* the format of the models, how the data for a subject were fit by 
a particular model. 
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Data Source * This analysis was based on the data for the 33 novices and 
the 33 experts who gave consistent answers (l#e. , high reliability between the 
two corresponding question sets). In order to provide an Intensive analysis of 
the performance of each subject in the sample, this analysis focused on only the 
simple problems • Simple problems are defined as those which contain symbols for 
number and/or plus and/or equals but which do not contain any multiplication 
symbols. Eighteen of the 44 problems in set A fit this description; these are 
problems -1 through 10 and 27 through 34 in Table 1. Thus, for each of the 33 
novices and 33 experts, the data source was a list of 18 answers for the 18 
simple problems* ^ 

Development of Models . The goal of the present analysis was to. develop 
simple production system models that would generate the performance of each 
subject on the 18 simple problems. Because of its efficiency and apparent 
relevance for the present task, a production system was used to represent the 
knowledge of each subject. A production system contains a list of productions 
with each production consisting of a condition and a corresponding action. 
Conditions, actions, and productions for the current problem are described In 



this section. 

The relevant conditions for the present analysis are related to having 
pressed one of the keys on the calculator keyboar4. The key relevant to the 
simple problems are the ten number keys (0, 1, 2, 3, 4, 5, 6, 7, 8, 9), the 
addition operation key (+) and the equals key (•). Table 2 shows nine con- 
ditions that could exist; three are not relevant to the simple problems since 
these conditions are never found in the problems. The other islx conditions 
contain an exhaustive ifst of the conditions present In the^slmple problems. At 
first blush, it might seem that Table 2 gives many redundant conditions and that 
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a alfflpller scheme Is to deal with only three basic condltlons—^presslng a 
number pressing a plus (+} , or pressing an equals ("]^^Hovever, the 
performance and comoients of subjects suggest that a key press means something 
different to some people depending on the imbtedlately preceedlng Icey press. 



Insert Table 2 about here 



The actions for each production consist of events that take place Inside 
the calculator. Following a system developed to represent the "Internal" - 
actions In computer Imguages (Mayer, 1979), each of the calculator actions can 
be represented as a triplet: some operation Is applied to some obj ect at some 
location In the calculator. The operations consist of the following: 

(1) Create — A number of expression Is placed In the display or 

register, e.g., when you press a number key that number^ 
appears In the display. 

(2) Destroy — A nuoiber or expression Is erased from the display or 

register, e.g., when you press the eqtials key the pre« 
vlous number In the display Is erased (and replaced 
with a new one). 

(3) Evaluation ~ An expression (from ths register) Is converted Into a 

single ^number, e.g., the evaluation of 3 + 2 Is 5. 
(For the current example, evaluation of a number or a 
number followed by an operation Is the number, e.g., 
evaluation of 3 Is 3, evaluation of 2 + Is 2). 
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The objects consist of: 

(1) Numbers — A number Is any single or multiple digit sequence 

such as 2, 14, 156, etc* 

(2) Operation ~ An operation Is a mathematical symbol for some arith- 

metic computation such as addition (+}, or multiplication 
(x). 

(3) Expression — ^ expression Is a sequence consisting of numbers and 

operations such as, 2 + 3 or 2 + or 2 + 3 x 4. 

The locations consist of: 

I 

(1) Display — The external display In a calculator normally consists 

of at least eight places, where each place can hold 
one digit. 

(2) Register — An internal register Is Inside the calculator and 

consists of subreglsters for Individual numbers and 
for operations. Expressions are held In the order of 
Input, with the first number on the left, followed by 
the operation, followed by the next number. 
Table 3 shows some typical actions that may occur for the simplj^ problems. 
It should be noted that the 12 actions listed In the table actually refer to 
groups of single actions. For .example, D • # consists of two single actions: 
erasing the old number from the display and replacing It with a new number. 
Also, it should be noted that the first five actions refer only to the display, 
and hi^t^^ effect on changing the register} the other actions refer only to 
the reglstiTvand have no effect on changing the display. 



X 
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Since Table 2 gives a list of all relevant conditions and Table 3 gives a 
list of relevant actions. It Is now possible to describe any production as one 
of the conditions coupled with one of the actions. For example, the production, 

P5. If # after + Then D » # and R » R + # 
means that when the number key Is pressed after the plus key the result Is that 
the old number In the display Is replaced with the new number, and the old 
expresclon In the register Is retained but the new number Is added to the 
right. Thus„ If the sequence had been 5 + 2-1- and now a 3 Is pressed, the 
display will contain a 3 (before there was a 2) and the register will contain 
the^^x^resslon 5+2 + 3 (before there was 5 + 2 +)• 

Fitting a model to each subject . The foregoing sections described that the 
data source for each subject was the 18 answers to the target questions and that 
models are based on (alternative) actions associated with each of the six 
conditions. Thxis, for the data of each subject the goal Is to develop a pro-* 
duction system which consists of six productions. 

The task was made somewhat easier by the fact that there are groups o€ 
subjects who gave identical answers for the simple problems. Based on their 
answers to the 18 simple problems, subjects could be grouped in one of six 
>dlstinct categories where subjects in each of the first five groups produced 
identical answers with one another. Group 1 contained 8 novices and 11 experts, 
all of «rtiom gave answers that were Identical to answers produced by inexpensive 
TI models. Group 2 contained 10 novices and 7 experts, all of whom gave answers 
that were identical to Grot^p 1 except for situations in which a plus key was 
•pressed. Group 3 contained 5 novices and 2 experrts, all of whom gave answers 
that were identical to Group 1 except for sitiiations in which a number key was 
pressed. Group 4 contained 2 novices and 4 experts, all of whom gave answers 
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that were similar to those produced by an Inexpensive Rockwell mod(2l. Group 5 
contained one novice and 3 experts, all of vhonr showed a mixture of acting like 
Group 1 and Group 4. Finally, there were 7 novices and 6 experts who gave 
Idiosyncratic patterns of answers for the simple problems. Some of the subjects 
In this category were very similar to one of the above groups except forgone or 
two minor deviations, while others seemed to have highly unique answers. In all 
cases, however, the answers were internally consistent within subjects as in- 
dicated by high correspondence between answers in set A and set B. The answers 
for each ol the common categories on each of the 18 simple problems are given in 
Table 4. . * ^ 

Models were gener>*ted for each of the common categories of answers and for 
each subject in the miscellaneous category. The production systems for each of 
the five common categories are given in Tables 5 through 9, respectively. 



Insert Tables 5 through 9^about here 

Groups 1, 2 and 3 all behave similarly but differ with respect to when an 
expression is evaluated and displayed. For example, consider the sequence of 
key strokes: 2 3 4 » • 

According to model 1, subjects think that the calculator evaluates ex- 
pressions only when a plus key (+) is pressed after a number or when an equals 
key (•) ±t$ pressed. Thus, .in the above example, the number in the display after 

each of the six key strokes will be, 2, 2, 3, 5, 4, 9. fhe 3 does not get added 

i 

to the 2 until a plus (or an equals) key is pressed; and the subtotal 5 does not 
get added to 4 until an equals' (or plus) key is pressed « 
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According to model? 2, subjects think that the calculator evaluates ex- 
pressions only when an equals («} key Is pressed. Thus, In the above example, 
the numbers In the display, after each of the six key strokes will be 2, 2f 3, 3, 
4, 9* The entire expression Is held In the register until an equals key ijs 
pressed* 

According to model 3, subjects think that the calculator evaluates ex- 
pressions as soon as a number key (#) Is pressed. Thus, In the above example, 
the numbers In tUe display after each respective key stroke will be: 2, 2, 5, 5, 
9, 9* The 3 gets added to the 2 as soon as the 3 Is pressed; the 4 gets added 
to the subtotal 5 as soon as the 4 Is pressed. ^ 

In summary model 2 Involves delayed evaluation (l«e*, nothing gets eval* 
uated until an equals Is pressed}^ model 3 Involves immediate evaluation (l*e*, 
expressions are evaluated as soon as possible), and model 1 involves a coin- 
promiss bstveen the two extremes (i.e., expressions are evaluated after a plus 
but not after a number key is pressed) • Another way to describe the differences 
among the first three groups is to say that group 1 treats a plus key like an^ 
equals, group 2 treats both a number key Ind a plus key like an equals, group 3 
treats neither like an equals. 

The fourth and fifth groups differ from the first three groups with respect 
to how to deal with the nlus key. The fourth group behaves as if the calculator 
has an automatic c5nstant«>-evaluation of an expression for a plus or an equals 
involves adding the number in the display to the number in the^register. This 
mode of evaluation is called "incrementing display" in the tables. The fifth 
group gives an "Incrementing display" only when two plus keys are pressed ^n 
sequence* Both groups are like group 1 in that they display the evaluated 
vsrslon*of the expression when & plus key or an equals key Is pressed but not 
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when a number key is pressed. The "incrementing display" characteristic of 
group 4, and to some extent group 5, is a more sophisticated and efficient 
feature of some calculators such as Rockwell. 




Insert Table 10 about here 



Table 10 gives a list of alternative actions associated with each of the 
first 6 .productions. As can be seen, the five common categories resulted in 
three 'alternatives for production P2, three for P4, two for F5, one for P6, two 

0 * 

for P7, And three for P8. ^ In the process of developing nsodels for the 14 mis- 
cellaneous subjects, several new alternatives were constructed as shown in Table 
10. Although a detailed analysis of the performance of each miscellaneous 
subject would require undue space, examples are given in this section. For 
example^ one of the \iovlces gives .the answers 2, 2, 5, 5, 12, 2, 0, 5, 5, 12, 0, 
0, 0, 3, 0, 0, 0, 3 for the 18 problems listed in Table 4,_respectlvely. This 
Subject seems to evaluate expressions immediately when a number key Is pressed, 
^rresponding to subjects* In group 3. However, in addition, this subject treats 
any irregular sequence of button presses (such as + after -* , or + after -) as a 
reset or clearing of the display and register. The productions which describe 
this subject's performance are: P2B, P4B, P5C, P6D, P7B and P8B. As another 
example, one of the experts gives tWe answers, 2, 2, 3, 5, 7, 2, A, 5, 10, 12, 
2, 4, 8, 3, 4, 8, 16, 7, for th^ 18 simple problems, respectively. This per- t 
formance Is similar to model 1 except that the display is incremented for « 
after +. The productions are P2A, P4A, P5A, P6A, P7A^, P8C. Modeis were fit to 
each of the miscellaneous subjects by taking the best fitting coimnon model 
(i*e*, models 1 through 5) and changing as few productions as necessary in order 
to nake the fit perfect. 
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/ The left side of the Table lO'lists^the f requencles^l^each production for 

tne experts and novices. TherQ is a tendency for experts to rely on productions 

which involve incrementing, the display, i.e., P5B and P8C. For example, these 

productions are used 6 times by novjfees'and 14 tl«K^s by experts. In addition, 

there is a tendency for experts; to rely on productions Vhich evaluate and dis-* 

play for + and « but not for #,,l.e., P2A, P4A, P7A; while novices tend to favor 

inane^te evaluation and display for #, i.ef, P2B, P4B, P7B. For example, the 

former set of productions is used 76 times by experts and 58. times by novices 

but the latti^r set is used 13 times by experts and 31 times by novices. 

* 

Analysis of Individual Subjects on Multiplication Problems * 

The previous section encouraged the idea that it is possible to describe 
the subject's "model of the calculator" for generating answer^ to 18 simple 
* problems. These analyses were based on problems containing only six possible 
conditions. In the present section^ we expand our analysis to include 16 ' - 
additional problems which contain multiplication. These are problems 11 through 

26 on Table 1. They provide three new conditions: x after # (i.e., pressing 

t 

the multiply key after pressing a number key, such as 2 x-. ), # after x (t»e., 
pressing a number key aftet pressing a multiply key, such as 2 x 3} and • after 
x (I.e., pressing an equal key after pressing a multiply key, such as 2 x «=X. 
Also, up to thi^spoint we have considered only conditions which Include two 
events, but this group of questions also allows (is to explore whether subjects 
use more than two e.-ents to determine chains of arithmetic; for example, if a 
subject evaluated all multiplications before additions; then 2^+ 3x7- would 
yield an answer of 23 but^lf a subject evaluated chains in order of presentation 
then the answer is 35. .Thus, this analysis will allow us to add three new 
piroductlons to each subject's model developed in the previousj section, and to 
O modify some productions for evaluating chain arithmetic. 
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For e^ch subjecti^ ve> asstme that the six productions established in the 
previous analysis still are operating for problems 11 through 26. Thus» the 
goal of the present section is simply to add three new productions (P10» Fll, 
P12) to the model of eacu subjecc. All 16 problems contain the condition x 
after #; almost all of the problems contain the condition, • after x; problems 
15, 17, and 23 provide the condition > after x; and several problems involve 
chain of + and operations such as problems 24 and 26. ^ * 

Table 10 shows the most common possible actions associated with each of the 
three new conditions (FIO, Pll, ?12), explored in this section. As with the 
analysis of simple problems, one issue concertvs when an expression is evaluated. 
If an expression is evaluated a& soon a& a multiplication operation (x) or an 
eq\ials (*) key is pressed, analogous to Model' 1 in tde previous analysis, then 
the productions selected woiil^be: PlOB (no evaluate for f), PllA (evaluate for 

x), and P12A (evaluate for equals). If an expression is evaluated only when an 

'A 

equals key is pressed, analogous' to the delayed evfluatien In Model 2, then the 
productions selected would be: FlOB (no evaluate for Fl]:B.(no evaluate for 
x), F12A (evaluate for >). If an expression is evaluated snr soon as a ntmber 
key is pressed, analogous to the Immediate evaluation of Model 3, then ttie 
productions selected would be: PlOA (evaj.uate for #) , PllB (no evaluate for x), 
P12B (no evaltiate for >). Finally « if st-b j ec^s^^iised an incrementing display for 
evaluating expressions and numbers as in Model 4 or 5 in the previous section, 
then the selected productions would be: PlOB (no evaluate for fllB (no 
evaluate for ^), and P12C (increment display for For purposes of this 
analysis we wjLll refer to each of these four clusters of three productions as 
)k>del Im, Model 2m, Model 3m, and Model 4?ii, respectively. 
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Table 11 sljovs the answers generated by each of the four multiplication 
models (i.t., productions PlO, Pll, and P12}. Thirty of the 33 novices and 23 
'^j^ of the 33 experts generated performance on the multiplication problems that was 
consistent with one of the four models; however, specific answers to some 
problems could differ from those listed in Table 11 in cases where different 
systems for evaluating chains of arithmetic or different productions for ?2 
through P8 were in use. The bottom of Table 11 shows the number of novices and 
experts who were fit by each of the four models. 

Insert Table 11 about here 

Model 2m allows for evaluation of a chain of arithmetic. Of the novices, 5 
subjects performed the arithmetic in order from left to right (as indicated in 
Table 11), one subject carried out additions before multiplications (e.g., 2 x 
3 f 7 ^ resulted In an answer of 20), one $*bject carried out multiplications 
bei^re ad(ti.tions (e.g., 2+3x7" resulted in an ^nswer of 23} and one carried 
out cominkatlpns only on the last two entries in the register (e.g., 2x3x7* 
resulted Ih 21; or 2 x 3 + 7 • resulted in 10). Of the experts, three out of 
five subjects showing model 2m performed chaitts from left to right, and two of 
the five experts performed multiplication before addition in a chain. 

There were also three miscellaneous novices and 10 miscellaneous experts. 
Of the novices f two subjecf^s gave model Im answers for problems that involved 
only numbers, multiplication, and/or equals but model 2m answers for problems 
with numbers, addition, multiplication, and equals. One novice gave model 4m 
answers for problems %rlth ntmibers, multiplication and/or equals but model 3m 
answers when problems involved multiplici<.tion, addition, numbers and equals. 
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Thus, these subjects behaved as If the conditions for actions depended on more 
than just the last two button presses. Ho additional productions were con- 
structed to try to fit this performance In Table 11. However » for tha 10 unique 
experts, several additional productions for PlO, Pll and P12 were constructed 
and are listed In Table 10. For example, one subject reset the display to zero 
for X after # and to no change for » after x. The productions for that subject 
are PlOB, FllC, P12A. Another subject Ignored the equal slgn^vhen It followed 
the mulclpllcatlon sign, as Indicated by productions PlOB, P11B# P12B.* Two 
other subjects reset t^e display for « after x giving the productions PlOB, 
PllB, P12C. Thus, many of the experts tend to add new productions for unusual 
button sequences; the effect of most of the new productions Is some sort of 
"resetlng** the display. The frequency of use of each alternative production for 
PlO, Pll, and P12 for all subjects Is summaxxzed In the left side of Table 10. 
Analysis of Indlvldusfl Subjects on Complex Problems 

Finally, the performance of each subject on problems 35 through 44 was 
analyzed. These problems contain many of the conditions already described In 
the previous tWo sections; thus. It was assumed that each sub/ ^t would use the 
twelve productions already determined by analyzing the first 34 problems In the 
test. However, problems 35 through 44 also contain four new conditions: x after 
«, X after x, + after x, and x* after +. Thus, In this section four new pro- 
ductions (P13, P14, P15 and.P16) are added to the model of each subject. 

Table 10 lists the alternative proouctlons for each of the four new con- 
ditions explored In this section. Table 12 gives some typical answers by 
suojects for problems 35 through 44. 



Insert Table 12 about here 
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Group 1 in Table 12 behave as if they were using productions 13A, 14A, 15A, 
and 16A along with delayed evaluation of expressions (based on earlier pro* 
ductiona). These subjects ^reat x after * and x after x as if there is no 
change t and if tvo consecutive operation Iceys are pressed (such as x after -*-) 
they use only Ithe last operation that was pressed. As shown in the bottom of 
Tablt 12y there vere ten novices and 15 experts who followed this procedure. 
There were also six more novices and one more expert in group 1*; these subjects 
behave as If they use the identical four new productions but they show immediate 
evaluation of expressions when the number key is pressed. In addition four 
novices (and no experts In group 2 behave like those in group 1 except that irfien 
there are two consecutive operations, the multiply "wins", i.e., for 2x+ the 
reg^ter stores 2x. The productions for this group are P13A, P14A, P15B, P16A. 
Similarly, two more novices and one expert in group 2' use the same four new 
productions as group 2 but act as if an expression is evaluated as soon as a 
nufld>er key is i>ressed. In addition, there were 2 novices and 4 experts in group 
3. These subjects treat the four new productions as if they serve to increment 
the dispUy. This procedure is indicated by the combination of P13A, PUB, 
P15C, P16C. These are the same subje(^cs who Increment the display for similar 

condition^ such as + after + or afte. ■ or • after +. 

i - 

There were also a large number of unique subjects~nlne novices and 12 
experts. Ho%rever, almost all of the subjects are closely related to either model 
1 or model 3, %rith just one production slightly different. For example, one 
subject is like taodel 1 except that the display is reset to sero for x after ^ 
or « after I X. The productions for that subject are P13A, P14A, P15D, P16D. 
Another sutjject has the same procedure as subjects in group 2 except that x 
after x resjults in the display being multiplied by the register; the productions 
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for this subject are P13A, PUB, P15B, P16A. For a third subject,, the display 
is incremented for - after x and x after - and when two consecutive operations 
are Input th^ multiply wins; the productions are, P13C, P14B, P15B, P16A. The 
left portion of Table 10 summarizes the frequencies of each of the alternative 
productions for all subjects on P13, P14, Pl5 and P16, 

General Conclusions 
Characterizing the Differences Among Subjects 

The present study suggests , that subjects differ with respect to their 
conceptions of the operation of electronic c^culators,. The foregoing analyses 
(sunnarlzed In Table 10) provided for a detailed description of the differences 
Bmon% subjects, with each subject being described as a list of productions. 
Hovever, the goal of this section Is to provide a more Integrated description of 
the major differences among subjects. Three basic kinds of differences were 
observed: <1) How Is an expression represented In the register? For example, a 
s#rlM of key smokes such as 2-hc3 can be represented as 2 4 3 or 2 x 3 or 0 or 
something else. (2) Uhen Is an expression evaluated? For example, does the 
calculator evaluate at the earliest possible opportunity such as 2 + .3 resulting 
in a display of 5, does the calculator wait for an equals to be pressed before 
an evaluation takes place, or does the calculator compromise between these two 
extreaes? <3) How is an expression evaluated? For example, a chain of arith- 
metic like 2 ^ 3 X 7 can be evaluated from left to right (answer Is 35) or with 
multiplication first (answer is 23) or in some other way; furthermore, non- 
standard sequences such as 2-f« can be evaluated by incrementing the display to 4 
or by ignoring the plus (display is 2) or by reset ing the display to 0. in this 
section, we explore how the subjects differ with respect to their conceptions of 
how to represent expressions, when to evaluate, and how to evaluate. > 
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Standard conditions ^ First, there are some general differences which 
emerge by Investigating differences' for standard conditions such as t after 4*, # 
after x, > aft^r x after #, > after #. These are sequences that follow the 
standard grammar of arithmetic, and are listed as P2, PlO, P4, Pll, and P7 In 
Table 10* 

The first Issue of how to represent expressions Is fairly straightforward 
for all subjects — symbols are added to the register in order from left to right • - 
For example, the keystrokes 2 4> 3 x-7 is represented in exactly that way in the 
register . 

•The second issue is when to evaluate the expression. In our analysis we 

« 

located three distinct approaches to the question of when to evaluate. The 
compromise method is to evaluate an expression whenever an equals key or an 
arithmetic operation key is pressed but not when a number key is pressed; the 
Innediate metlM)d is to evaluate as soon as a number key is pressed (e.g., for 3 
4> 5 display shows 8); the delayed method is to evaluate only when an equals key 
is pressed (e.g., for 3-^5 the display shows 5). The novices and experts tend 
to differ with respect to their consensus on when to evaluate. Of the novices 
13 tend to opt for compromise evaluation, 13 for delayed evaluation, and 7 for 
immediate evaluation; for experts there is a much stronger consensus of 24 
subjects favoring compromise evaluation with 7 favoring delayed evaluation ^nd 2 
favoring immediate. ; A c hl^square test revealed that novices and experts differ 
significantly with respect to the proportion of subjects fd^oring compromise 
evaluation, ■ 6.15; df • 1, £ < .05. 

The third issue concerns how to evaluate an expression. In most cases, 
subjects overwhelmingly follow th^ normal rules of- arithmetic. However, as 
not#d earlier, when subjects use delayed evaluation they may be confronted with 
a chain of arithmetic such as 2 4* 3 x 7. While the majority of subjects eval- 
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uate a chain from left to right (I.e., generating an answer of 35), some experts 
carry out multiplication before addition (i.e., answer is 23), and some novices 
use other schemes such as carry out additions before multiplications or carry 
out only the last computation (i.e., answer is 21). 

Non-standard conditions (equals after operation) . In the present study we 
also Investigated subjects* interpretations of several non-standard conditions, 

such as « after 4- or after x. These are conditions which violate the grammar 

« 

that demands a number between the operation symbol and the equals symbol. Table 
10 represented these as PS a ad P12. 

The main issue here is how to represent and evaluate an expression when, the 
last key press was an operation and now an equals is pressed. For production 
P8, the majority of novices (n » 27) and r.he majority of experts (n » 25) ignore 
the last 4- key that was pressed. Thus, a sequence like 24« results in a display 
of 2, or a sequence like 24->4« results in a display of 5. We call this the "no 
effect'* approach because subjects act as if the plus key had no effect. A 
second approach is what we call the "Incrementing" approach; here subjects 
create some number to go between the + and the • such as the number in the 
display. For example, the sequence 24* results in a display of 4 (i.e., it is 
treated as 2-l-2«), or 2-f3+« may result in 10 (i*e. , it is treated as 5+5«) or 8 
(i»e., it is treated as 2+3+3«). There were three novices and 6 experts who 
opted for the incrementing approach. A third option is what we call the "reset'^ 
approach. Here subjects reset the display to some number (such as zero) for any 
non-standard sequence of key presses. Three novices and two experts used a 
versltm of the reset approach. The comparable figures for production Pl^ were 
29 novices and 25 experts favorjsd the no effect approach while 4 novices and 8 
experts favored the incrementing approach. Although the proportion of subjects 
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favoring the Incrementing approach isk twice as high for experts as for novices ^ 

cM-equare tests failed to Indicate that the proportion of Inctementlng subjects 

was greater for experts In P8, ■ >*lt *f " It or P12, ■ '92, df • 1, 

t fon-standard conditions (|tvo consecutive operations) > Another type of non- 

i _ 
standard condition Investigated In this stti4y was two consecutive operations 

such as + after x after ^ after x, or x after +• These are conditions 

which violate the gramnatlcal demand for a number between any two operation 

symbols. Table 10 presents these as PS, PlA, PIS, P16. 

^ The main issue here is how to represrat and/or evaluate an expression when 

the last key presses are two arithmetic operators. The three major options 

chosen by our subjects correspond to those discussed above: "no effect'' in- 

vclves selectiJig one of the two operation signs to be Included in the register 

and ignoring the other; for example , the most. common version of this approach is 

to ignore the second operation so that 24-h is represented in the register aa 

2+ or 2xx is represented as 2x. "Incrementing" involves selecting a number to 

be inserted between the operator symbols; the most coomon version of this 

approach la to Insert the number from the display so that 2-H- becomes 2+2+ or 

2xx becomes 2x2x* "Reset" involves clearing the display such as setting it to 

zero; for example » 2++ results in 0 being displayed. For production P5 the 

majority of novices (n « 28) and experts (n » 24) opted for the no effect 

approach; in addition four novices and nine experts opted for the incrementing 

« 

approach; and one novice and no experts reset the display. The figures for 

0 

production P14 are similar: 28 aovices and 24 experts opted for no effect; 
incrementing was opted for by 4 novices and 8 experts; and one novice ahd one 
expert opted for the reset approach. The patterns for tlS'and P16 are similar — ^ 
the majority of each novices and experts opt for no effect but a substantial 
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number of experts opt for the Incremi^ntlng option. In all productions, the 
proportion of experts who increment is more than twice that of the novices. 
However, even in the most extreme case, the differences in proportion of "in- 
crementers'* between experts and novices fails to reach statistical significance, 
• 2.27, df • 1. 

Hon^'Standard conditions (operation after equals) . Finally, our Cest in-* 
volved two productions P6 and P13, which involve + after « and x after • re* 
spectively. The status of the register after an - key is pressed is that it 
contains a number. Thus, these non-standard conditions are most frequently 
treated in the same way that after # or x after # is treated. For P6, 30 of 
the novices and 32 of the experts use this "no effect" approach of simply adding 
a plus sign to the register. For P13, 28 novices and 27 experts follow the "no 
effect*' approach of adding a multiply sign to the register. However, a sizable 
minority of the experts (n • 6) opt for an incremating approach while a sizable 
minority of the Novices opt for a reset, option (n « 4). 

Sunnary . The present study provides new information concerning how httmans 
think about calculators. First, we were able to apply .the analytic techniques 
of cognitive psychology to a real-*world domain. This allowed a formal and 
detailed description of how each subjctct Interpreted what was going on'' Inside 
the "black box" when a key ^ was pressed. Second, in spite of the fact that all 
of our subjects were proficient in using a calculator to ftolve standard compu- 
tational problems, we observed tremendous individual differences among users in 
their Interpretations of the logic of the calculator's operating system. Thus, 
in epite of apparent similar performance on standard problems, people differ 
greatly in their knowledge of how the calculator solves problems. 
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Experts tended to give more consistent ans^rs, as would be expected; 
howaver. they also tended to prefer certain operating characteristics su«h as 
evaluating an expression for either ah operation key or an .equal key (compromise 
•valuator), and Incrementing the display during evaluation with non-standard 
conditions. Further work Is needed to determine whether people with certain 
••ts of intuitions can use their calculators more creatively or can learn a new 
computer language (such as progrannable calculators or BASIC) more efficiently 
than people with other sets of Intuitions. In addition, future work Is needed 
to determine whether intuit lons-once they have been diagnosed-can be altered 
through instruction. It is hoped that the groundwork laid in this study will 
serve as an Incentive for continued work in the development of a theory of 
computer literacy. 
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for study 2. 

3 

For each problem in which the subject gave inconsistent answers between 

>^ 

set A and set B, the answer to th# A set was used unless it was inconsistent 
with related items. 

* 

Since the'^HP-Zl uses reverse Polls^ notation (RFN) it was not used as a 
model in. this study. 



1 

2 
3 
4 

5 . 

6 

7 

8 

9 

10 



11 
12 
13 
14 
15 
16 



Problem 
2 

2+ 
2+3 
2+3+ 
2+3+7 

2+- 

2+3- 

2+3+- 

2+3+7- 



2x 

2x3 

2x3x 

2x3x7 

2x- 

2x> 



Table 1 

Freq|«eneles°(i}f Answers for 44 Problems 

Study 1 
Modal Answer Alternatives 

(2)-33 

(2) -33 

(3) -25 
(5)-21 
(7)-26 
(2)-33 
(2)-29 
(5)-33 
(5)-30 

(12)-29 



(2) -33 

(3) -24 

(6) -24 

(7) -24 
(2)-29 
(6)-33 



(5)-8 

(3) -12 

(12)-6;(15)-1 

(4) -3;(0)-l 
(10) -3 

(10)-2;(11)-1(15)-1 



(6)-9 
(3)-9 
(42)-9 
' (4)-4 



Study 2 

Modal Answer Alternatives 



(2)-33 

(2) -33 

(3) -32 
(5)-23 
(7)-32. 
(2)-33 
(2)*=27 
(5)-33 
(5)-26 

(12)-33 



\ 



(2) -32. 

(3) -33 

(6) -23 

(7) -32 
(12)-25 

(6)-33 



(0)=1 
(5)-l . 
(3)-9;(0)-l 
'(12)-1 

(10)-5;(3)=1;(8)«1 



(0)-l 

(3) -9;(0)-l 
(6)-l 

(4) -7;(0)-l 



6 

Table 1 (Con't.) 









Frequencies of Answers for 44 Problems 




- 

Problem 






acuuy J. 




Study 2 


Nuaber 


Problein 


ffodal Answer 


Alternatives 


Hodal Answer 


Alternatives 


17 


2x3x- 


(5)-29 


(36)-4 


(6)-23 


(36)-6; (3)-2: (18)-1: (0)-l 


18 


2x3x7- 


(42)-31 


(21)-1;(7)-1 


(42)-32 


(41)-1 ' 


19 


2+3x 


(5)-22 


(3)-ll 


(5)-18 


(3)-13;(0)-l:(6)-l 


20 


2+3x7 


(5) -25 


(35)-7;(42)-l 


(7)-32 


(42)-l 


21 


2x3+ 


(6)-24 


(3)-9 


(6)-25 


(3)-7;(0)-l 


22 


2x3+7 


(7)-26 


(13)-7 


(7)-32 


(13)-1 


23 


, 2+3x- 


(5)-26 


(O^^m'K* CWmO* t'\^\m'\* (0\m'\ 




(25)-3; (3)-3; (ll)-2; (2)-l; (15)-1 


2A 


2+3x7'" 


(35)-29 


(23)^2; (21)-1;(42)-1 


(35)-23 


(23)-7;(42)-2;(45)-l 


25 


2x34- 


(6)-29 






(12)-5;(3)-l;(18)-l 


26 


2x3+7- 


(11^-29 


f42)-2: flO^»l* 




ai)-l;(42)-l 1 


27 


2++ 


(2)-29 


(4)-3:(0)«l 




(4)-9;(0)»l S 


9ft 






(4)-4;(0)-2 


(2)-25 


(4)-8; (0)«1 g 


29 4 o 


2+-+-+ 


(2)-26 


(8)-3;(0)-2;(4)-l;(6)-l 


(2)-23 


(8)-5;(4)-2;(6)-2;(0)-l o p 


30 


2+-+3 

« 


(3)-28 


(5)-5 ' 


\ (3) -32 


(2)-l 1 


31 


2++- 

* 


■(2)-27 


(40-3;(8)-2;(O)-l 


(2) -22 


(4)-6;(8)-4;(6)-l g 


32 


2-W- 


(2)-28 


(8)-2;(4)-l;(6)-l;(0)-l 


(2)-24 


(8)-5;(4)-3;(6)-l 
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Table 1 (Con't.) 
Frequencies of Answers for 44 Problems 



Problea 






Study 1 


Study 






Huabcr 


■ Problea 


Modal 


Answer Alternatives 


Modal Answer 


Alternatives 




33 




(2)-28 


(16)-3;(8)-l;(6)-l 


(2)-24 


(16)-5;(6)-3;(8)-l 




34 


2+-f3- 


(5)-26 


(7>4;(3)-3 

• 


(4)-26 


(7)-6;(3)-l 

- 




35 


2xx 


(2)-29 


(4)-3;(0)-l p 


(2)-23 


(4)-9;(0)-l 




36 


2x-x 


(2)-28 


(4)-4;(0)-l 


(2)-24 


(4)-8;(0)-l 




37 


2x-x-x 


(2)-27 


(16)-3;(4)-l;(8)-l;(0)-l 


(2)«21 


(l6)-5;(8)-4;(4)-2;(0)-l' 




38 


2x-x3 


(3)-24 


(6)-9 


(3)-29 


(6)-3;(12)-l 




39 


^xx- 


(2)-26 


(4)-4;(8)-l;(0)-l;(16)-l 


(2)-20 


(4)-7;(16)-3;(8)-2;(0y-l 




40 


2x-x- 


(2)-27 


(16)-3;(0)-l;(4)-l;(8)-l 


(2)-22 


(16)-5;(8)-3;(4)-2;(0)-l 




41. 


2x"x"X" 


(2)-27 


(256)-3(8)-l; (16)-1;90)-1 


(2)-21 


(256)-5 ; (16)-2; r8)-3; (3Z)-1; (0) 


-1 


42 


2x-x3- 


(6)-24 


(3)-5;(12)-3;(48)-l^ 


(6)-24 


(12)-7;(3)-l;(0)-l 




43 


2x+3- 


(5)-16 


(6)-13;(7)-2;(8)-l;(3)-l 


(5)-21 


(7j-6;(6)-3;(e)-2;(3]i-l 


H» 


44 


2+x3- 


(6)-25 


(5)-4;(3)-2;(l2)-2 , 


(6)^24 


(12)-6;:5)-l;(7)-l;(e)-l 


ft 
H* 
O 



g 



tlotia, - Number in parentheses indicates answer; number to right of equals indicates frequency. 



n 

p. 

n 
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Table 2 



Conditions for Simple Problems 



Naac 

PI 

P2* 

P3 

P4* 

P5* 

P6* 

P7* 

P8* 

P9 



Condition 

I after # 
I after + 
# after - 
+ after I 
+ after + 
+ after - 

- after # 

- after f 
■ after ■ 



Example 

2 3 
+ 3 
- 3 
+ 
+ 
+ 



2 
+ 

3 
+ 



Description . 

Pressing a number key after pressing a number key. 
Pressing a number key after pressing a plus key. 
Pressing a number key after pressing a equals key. 
Pressing a plus key after pressing a number key. 
Pressing a plus key after pressing a plus key. 
Pressing a plus key after pressing an equals key. 
Pressing an equals key after pressing a number key. 
Pressing an equals key after pressing a plus key. 
Pressing an equals key after pressing an equals key. 



0 
ft 
c 

rt 



Note. — Asterisk (*) indicates that production is relevant to simple problems. Pl» P3 and P9 are not 
relevant since they do not occur in th^ simple problems. 
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, ^ Table 3 

^ Some Actions for JSlmple Problems 

Action Description • * 

V 

4 

D • D No change In the display 

D « I Erase the old number from the display. Put the new number In the display. 

D R Erase the old number fr^ the display. Copy the number from the register Into the display. 

D > eval(R) Erase the old number from the display. Put the value for the expression In the register Into the display. ^ 

D - evaKDfR) Erase the old number from' the display. Replace It with the value for the sum of that number from the 

display and the value In the register. 

U «* R No chtinge In register. 

R ' f Retain the present exprest^icn In the register. Place a number to the right of the expression In the register. 

R « "R-l-l" Retain the present expression in the^reglster. Place a number to the right of the expression in the register. 

R " eval (R) Erase the old number or expression from the register. Replace it with the evaluation of the number or 

expression. 5* 
c 

R - eval (IH) Erase the old number or expression from the register. Replace it with the sum of the number in the ___ « 



R - eval (R)+ Erase the old expression or number from the register. Replace it with the evaluation of that number or & 

o 



display plus the evaluation of the register. S 



expression, and follow that with a plus. ^ 



ft 

£ 



o 

m 



er|c 



Problem 

Muaber Problem 
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Table 4 

Problems and Answers or 18 Simple Items for Four Groups of Subjects 

Group 5 
' J^swer Miscellaneous 



Group 1 
Answer 



Group 2 
Answer 



Group 3 
Answer 



Group 4 
Answer 



1 


2 


2 




2 


2 


2 
















2 


2+ 


2 




2 


• 2 


2 


3 


2+3 


3 




3 


5 


3 


4 


2+3+ 


5 




'3 


5 


5 


5 


'2+3+7 


7 




. 7 


12 


7 


6 


2- 


2 




2 


2 


2 




24- 


2 




2 


2 


4> 


sX 


2+> 


5 




5 


5 


5 
















9 


\ : 2+3+- 


5 




5 


5 


10 












* 




10 


\ 2^3+7. 


12 




12 


12 


12 


27 




2 




2 


2 


4 


28 


2+-f\^ 


2 




2 


. 2 


4 


29 


2I-I"i 


2 




2 


2 


8 


30 


2+iM-3 


3 


t 

£ 


3 


^ 5 


' 3 


31 


2+4- 


2 




2 


2 


8 


32 


2+»H» 


2 




2 


2 


8 


33 




2 


•4 


2 


' 2' 


16 


34 


^ 2+-+3- 


5 




5 


5 


7 



2 
2 
3. 
S 
7 
2 
2 
5 
S 

12 
4 
2 
2 
3 

. 4 
2 
2 
5 



Nuabar of Subjects- 
Study 1 



8 



HiabM of Subjects- | 
Study 2 11 



10 
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Table 5 

Production System for Model 1, (Compromise Evaluation) 



Production 
Kame 


Condition 


Action 












* 




Comments 




P2A 


If # after + ^ 


then 


Set 


D « 


# 




Set 


R 




"R + #" 






P4A 


If + after 1 


then 


Set 


D - 


eval 


(R) 


Set 


R 


s 


* 

eval (R) + 






PSA 


If + after + 


then 


Set 


D - 


D 




Set 


R 


+ 


» R + 


(NO CHANGE) 




P6A 


If + after - 


then 


Set 


D - 


eval 


(R) 


Set 


R 


as 


eval (R) 






P7A 


If - after # 


then . 


Set 


D - 


eval 


(R) 


Set 


R 




eval (R) 






PSA 


If « after + 


• 

then 


Set 


D - 


eval 


(R) 


Set 


R 




eval (R) 




























- ^ V 


1 



Note.' — Evaluates after + or -. Display Is non-lncrementlng. 
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Table 6 



Production System for Model 2 (Delayed Evaxuatlon) 



Production 

P2A 
P4B 
PSA 
P6A 
!»7A 
PSA 



■ Condition 
If # after + 
If + after # 
If after + 
If + after « 
If « after # 
If » after + 



Action 



then ^dt D = 



then ^Set D 
then Set D 



D 
D 
D 



then Set D 
then Set D - eval (R) 
then^ Set D = eval (R) 



Se. R = "R + //•• 
Set R » evai (R) + 
Set R + = R + 
Set^ R = R + 
Set K ^ evai (R) 
Set R = eVal (R) 



CoBBnent s 

(SAME AS MODEL 1) 

(SAME AS MODEL 1) 

(SAME AS MODEL 1) 

(SAME AS MODEL 1) 

(SAME AS MODEL 1) 



Note. — Evaluates after Display Is non- Incrementing. 



ON 



rf 
C 
H* 
rf 

s- 

S 
I 

c 

ft 

£> 

O 
C 

E* 

rf 
O 
n 



9 



Production 
Name ^ 

P2B 

P4B 
PSA 
P6A 
P7B 
P8B 



Table 7 

Production System .for Model 3 (Iimpedlate Evaluation) 



Condition 
If # after + 
If + after # 
If + after + 
If + after * 
If » after # 
If - after + 



Action 



then Set D - eval (R +#) Set R « eval (R-l-#) 



then Set D « D 

then Set D-D 

then Set D » D 

then Set D » D 

then Set D « D 



Set R = eval (R) + 
Set R = R + 
Set R « R + 
Set R = R 
Set R « R 



Comments 



(SAME AS MODEL 1) 
(SAME AS MODEL 1) 
(NO CHANGE) 
(NO CHANGE) 



Note. —Evaluate after 9. Display Is non-lncrementlng. 
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Production 
Nimber 

P2C 

P4C 

P5B 

P6A 

P7A 

P8C 



Table 8 

Production Syatem for Model 4 (Incrementing Display) 



Condition 
If I after -f 
If + after # 
If + after + 
If + after - 
If - after # 
If - after + 



Action 

then Set D « # Set R - eval (R + #) 

then Set D - R Set R » R 

then Set D * eval (I>fR) Set R = eval (D+R) 

then Set D = eval (R> Set R « eval (R) 

then Set D « eval (R) Set R » ^val (R) 

then Set D » eval (I>fR) SEt R = Jjval (I>fR) 



Note, — Evaluate after + or «. Display is incrementing. 



Comments 



(INCREMENTING DISPLAY) 
(SAME AS MODEL 1) 
(SAME AS MODEL 1) 
(INCREMENTING DISPLAY) 



c 



f 



4> 
00 



Table 9 

Production System for Model 5 (Partially Incrementing Display) 

Production 

Name Condition Action 



P2A 


If 


1 after + 


then 


Set 


D 








Set 


R 




+ #" 


P4A 


If 


+ after # 


then 


Set 


D 




eval 


(R) 


Set 


R 




eval (R) + 


P5B 


If 


+ after 


then 


Set 


D 




eval 


(IHR) 


Set 


R 




eval (D+R) 


P6A 


If 


.+ after « 


then 


Set 


D 




eval 


(R) 


Set 


R 




eval (R) 


P7A 


If 


» after # 


then 


Set 


D 


ae 


eval 


(R) 


Set 


R 


m 


eval (R) 


P8A 


If 


» after + 


then 


Set 


D 


8 


eval 


(R) 


Set 


R 




eval (R) 



Note.— Compromise between model 1 and model 4* 



cu 



Frequency 
In Study 1 

21 



Table 10 

Frequencies of Productions for All Subjects 



18 

4 
0 



62 



28 



Frequency Production 

in StudjL2 Nunber Condition Action 

27 P2A 



18 



4 
2 

24 



P2B 



P2C 



P2D 



PSA 



If # after + then Set D - #, 

Set R = "R+#" 
If# after + then Set D - eval (R+#V 

Set R - eval (R+#) 
If I after + then Set D - # 

Set R - eval (R+#) 
If # after + then Set n « gyal (R+D), 

Set K ^ eval (R+D) 



Description 

Delayed evaluation and display 

Inniedlate revaluation and display 

Inmedlate evaluation and delayed display 

Immediate evaluation and display vlth 
Incrementing evaluation 



P4A If + after t 

If + after # 

P4C If + after # 

P4D If + after # 



then Set D « eval (R), 
Set R -eval (R) + 

then Set D-D, 

Set R » eval (R) + 

then Set D « R, Set R-R+ 

then Set D » 0, Set R » 0 



Immediate evaluation and display 



M 
9 
ft 
C 

ft 

10 



Immediate evaluation and delayed display ^ 



Delayed evaluation and display 
Reset to zero 



If + after + 



then Set D-D, 
, Set R+ - R+ 



ft 
n 

O 

c 

M 

rr 
O 

(A 



No change 



\ 



Table 10 (continued) 

Frequency Frequency Production 

In Study 1 In Study 2 Number Condition Action 

4 ' 9 P5B If + after + then Set D « eval (DfR) 

Set R • eval (I>fR) 
1 . 0 P5C If + after + then Set D « 0, 

Set R » 0 



Descrlptlcn 

Insnedlate Incrementing evaluation and 

display 
Reset to 0 



30 



ERLC 



32 


P6A 


If 


+ after 


- 


then 


Set 


D 




D, 
















Set 


R 




R+ 




0 


p6B 


If 


+ after 




then 


Set 


D 




D. 
















Set 


R 




eval 




0 


P6C 


If 


+ after 




then 


Set 


D 




0. 
















Set 


R 




R+ 




0 


P6D 


If 


+ after 




then 


Set 


D 




0. 
















Set 


R 




0 




1 


P6E 


If 


+ after 




then 


Set 


D 




D. 
















Set 


R 




eval 


(#+D) 
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No change In display, add -i- to 

expression In register 
No change In display. Immediate 

Incrementing evaluation 

< 

Reset display to 0, add -I- to 

expression In register 
Reset display and register to 0, 



0 
ft 



1 1 

evaluate sum of constant and value ^ 

K ' CO 

^ In display ^ 

o 

No change In display g 



Ui o 
CO 



1 



65 



Table 10 (continued) 



Frequency Frequency Production 

In Study 1 In Study 2 Number Condition Action 



26 


31 


P7A 


If - after # 


then 


Set D 




eval 


(R), 












Set R 


- 


eval 


(R) 


6 


2 


P7B 


If » after # 


then 


Set D 


cs 
















Set R 




R 




1 


0 


P7C 


If, a after 9 


then 


Set. D 




R» 














Set R 




R 




21 


23 


P8A 


If * after + 


then 


Set D 


s 


eval 


(R), 












Set R 




eval 


(R) 


6 


2 


P8B 


If « after + 


then 


Set D 


XX 
















Set R 


- 


R 




2 


5 


P8C 


If - after + 


i 

, then 


Set D 


a 


eval 


tlw-R) 












Set ,R 


IS 


eval 


(W-R) 


2 


0 


PSD 


If - af tw + 


then 


Set D 


a 


eval 


(R). 


6:; 










Set R 




0 






1 


PSE . 


If - after + 

* 


then 


Set D 


*e 


eval 














Set R 


a 


R 





Descrlptlgn 

Imroedlate evaluation and display 
No change 

Display value In register 



Immediate ev^lua^j^n and display 
No change In display or register 

Immediate Incrementing evaluation 

and display ' * 

Display the evaluation of the expresslpn 

In the register, reset the 

register to 0 b* i 

Display the sum of the value In the 

display plus a constant , no change 



ERLC 



In register 



Table 10 (continued) 



Frequency Frequency Production 

In Study 1 in Studjr 2 Number ^Condition Action 



25 



11 



19 



19 



ERIC 



27 



15 



16 



18 



6u 



P8F 



If « after + then Set D « D, 
Set ^ 0 



Description ^ 

No change in display » £et register to 0 



PlOA 


If 


# after x 


then Set 


D = 


eval 


(R*//) 


Immediate evaluation and display 








Set 


R = 


aval 


(R*//) 




PlOB 


If 


It after x 


then Set 
Set 

• 


D «= 
R =» 






Delayed evaluation and display 



PllA If X after if then Set D - evcl (R) , 

Set R = eval (R)* 

PUB If X after # then Set D « D, 

Set R - eval (R)* 

Pile If X after t then Set D » 0, 

Set R « R* 

(other mixed) 
— ~~~*"~~'"'~'"~***~*~~~~"*~~~~~ 

P12A If - after x then Set D - eval (R) , 

Set R - eval (R) 



Immediate evaluation and display 



Immediate evaluation and no change in 

ft 

display - g 

Delayed evaluation and reset display to 0 ^ 

o 
c 
ft 



s 

ft 
O 

Cft 



Immediate evaluation and display 



6D 



Table 10 (tfontlnued) 

Frequency Frequency Production 

in^£^ ln_Stud^ Nu5ber_ Condition Action 

P12B If = after x then Set D = D, 

Set R = R 

rf = after x then Set D - eval (D*R) , 

Set R = eval (D*R^ 
If » after x then Set D = eval (D*R) 

Set R = eval (R) 
If = after x then Set D = eval (R*d), 
Set R = eval (R)* d 

(other mixed) 



P12C 



PI 2D 



P12E 



Description 
No change 

Wdlate Incrementing evaluation and 
display 

Inmedlate Incrementing display. Wdlate 

evaluation for register 
Immediate evaluation and display In the 
constant Increment 



28 

4 
1 



27 

0 

3 



P13A 

P13B 
P13C 

PI 3D 

P13E 



If X after - then Set D = u, 
Set R = D* 
If X after - then Set D = 0, Set R = 0 
If X after - then Set D = eval (D*R) , 

Set R » eval (D*R) 
If X after - then Set D - eval (D*R) , 
Set R - R 

If X after - then Set D - R, 

Set eval (D*R) 



Delayed evaluation and display 
Reset to 0 

« 

Immediate Incrementing evaluation and 
display 

inmiedlate Incrementing display, no change 
in register y ^ 

Immediate Incrementing evaluation, 
delayed display 



0 
ft 



o 

CO 



0) 



n 
c 



o 

0} 
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Table 10 (continued) 



Frequency Frequency Production 

in Study 1 In Study 2 Ntun ber Condition Action 



28 



19 



10 



24 



P14A If X after x then Set D = D, 

Set R = R* 

# 

P14B If X after x then Set D = eval (D*R) 

* Set R = eval (D*R)* 

P14C If X after x then Sec D - 0, 

Set R => 0 

P14D If X after x then Set D « O, 

Set R » D*R 



25 



Description 

Delayed evaluation and display 

Ingnediate incrementing evaluation and 

display 
Reset to 0 



Inmediate incrementing evaluation, 
no change in display 



P15A If + after x then Set D « D, 

Set R » R+ 

P15B If + after x then Set D « D, 

^ _ Set R » R* 

P15C If + after x' then Set D = eval (0*R) , 

Set R - eval (D*R) 
P15D If + after x then Set D ■ 0, 

Set R - 0 , 

F15E If .+ after x then Set D - eval (D*R), 

Set R - eval (D*R)+ 



ERIC 72 



Set register sign from * to + 
No change 

Inmediate incrementing evaluation and 

display 
Reset to 0 

Immediate incrementing evaluation and 
display with register •ign to + 



0 
ft 

c 



0 

m 



o 



Table 10 (Continued) 

Frequency Frequency Production 
in Study 1 in Study 2 Number Condition Action • ^ Description 



28 


25 


P16A 


If 


X 


after + 


then 


Set 


D 




D, 




Set register sign from + to * 








- 








Set 


R 




R* 






1 


0 


P16B 


If 


X 


after -I- 


then 


Set 


D 




D, 




No change 
















Set 


R 




R+ 






2 


4 


P16C 


If 


X 


after 


then 


Set 


D 




eval 


(D*R), 


Immediate incrementing evaluation and 
















Set 


R 


ss 


eval 


(D*R) 


display 


2 


1 


P16D 


If 


X 


after 


then 


Set 


D 




0, Set R « 0 


Reset to 0 


0 


3 


P16E 


If 


X 


after + 


then 


Set 


D 




eval 


(D*R) , 


Immediate incrementing evaluation and 
















Set 


R 




ev 


/^D*R)* 


display with register sign set to 




7-; 



ON 



ERIC 



Intuitions About Calculators 

Table 11 

ProbletDS and Answers for 17 Multlpllctlon Items by Four Groups of Subjects 



Problem 
nuiiiDer 


Problem 


Group Im 
Answer 


Group 2m 
Answer 


Group 3m 
Answer 


Group 
Answer 


1 1 

XX 


2 


X 












x^ 


2 


X 


3 






0 




IJ 


2 


X 


3 X 


0 


o 

J' 


0 


0 


Xh 


2 


X 


3x7 


/ 


7 


42 


7 


Xj 


2 


X 


- 






2 


4 


lo 


2 


X 


3 - 


0 


0 


0 


0 


17 


2 


X 


3 X - 


0 


0 


0 


3d 


1 o 


2 


X 


3x7- 






42 


42 


1 o 

X7 


2 


+ 


3 X 


c 




!) 


e 


20 


2 


+ 


3x7 


7 


7 


35 


7 


21 


2 


X 


3 + 


6 


3 


6 


6 


22 


2 


X 


3 + 7 


7 


7 


13 


7 


23 


2 


+ 


3 X - 


5 


5 


5 


25 


o 24 


2 


+ 


3x7- 


35 


35 


35 


35 ^ 


25 


2 


X 


3 + - 


6 


6 


6 


. 12 


26 


2 


X 


3 + 7 - 


13 


13 


13 


13 


Number of Sub1r.cts-Study 1 


11 


8 


7 


5 


Nunbcr of Subjects-Study 2 


15 


5 


0 


3 



Miscellaneous 



3 
10 



76 



ERIC 



Problem 
Number 

35 

36 
37 

38 

39 

40 

41 

42 

44 



Intuitions About CalculAcors 
58 

Table 12 

Problems and Answers for 10 Complex Items by Five Groups of Subjects 



Problem 
2 X X 
2 X ■ X 

2 X ■ X ■ X 

3 X ■ X ■ 3 
3 X X ■ 

2 X ■ X ■ 
2 X • X ■ X 
2 X - X 3 - 
2 X + 3 - 
2 + X 3 - 



Number of SubJects-SCudy 1 
Mumber of Subjects-Study 2 



10 
15 



4 



Group 1 Group 1' Group 2 Group 2' Group 3 

Answer Answer Answer Answer Answer 
2 2 
2 2 

2 2 

3 6 
2 2 
2 2 
2 2 
6 ' 6 

5 5 

6 6 



2 
2 
2 
3 

2 
2 
2 
6 
6 
6 

4 
0 



16 
3 

16 
16 
256 
12 

12 

2 

4 



Miscellaneous 



9 

12 
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